Valutazione dell’Indice di Overlapping (η) tramite Simulazione: Un Confronto con gli Effect Size Tradizionali
Perugini, A., Calignano, G., Pastore, M.
AIP Sperimentale 11 - 13 September 2025 | Torino
Effect sizes for comparison
The Overlapping Index
The overlapping (\(\eta\)) is an index of effect size and varies from 0 and 1.
![]()
It is \(0\) when the two distributions are completely disjoint and \(1\) when there is complete overlap.
Common Language Effect Size (CLES)
It express the probability that an observation randomly extracted from one group will be higher than the one extracted from the other group.
Study Design
n = 10, 50, 100, 300, 500, equal in the two samples.
Mean difference: \(\delta\) = 0, 1, 2;
Variability: \(\sigma\) = 1, 3, 5;
Skewness: \(\alpha\) =0, 5;
For the total combination of \(5 × 3 × 3 × 2 = 90\) conditions we generated \(500\) sets of data on which we performed the analysis.
Results
We compared the two effect sizes on the following indexes of bias:
The Absolute mean bias (AMB)
The Relative mean bias (RMB)
The Mean Square Error (MSE)
Absolute Mean Bias
![]()
Absolute Mean Bias
![]()
Relative mean bias
![]()
Relative mean bias
![]()
Mean Square Error
![]()
Mean Square Error
![]()
Discussion
The Overlapping index presents less Relative Mean Bias (RMB) compared to the CLES in conditions with no homogeneity of variance between the two groups.
Next Steps
We will compare the Overlapping Index to other Effect Sizes but the problem of differences in scale needs to be solved.
Results
We compared the two effect sizes on the following indexes of bias:
The Absolute mean bias (AMB): obtained as the mean of the absolute bias: \(1/B\sum_b (\hat{\theta}_{bj}-\theta_j)\) in which \(\theta_j\) is the value in the condition \(j\) and \(\hat{\theta}_{bj}\) the one estimated in the replication \(b\) in the condition \(j\).
The Relative mean bias (RMB): obtained as the mean of relative bias: \(1/B\sum_b (\hat{\theta}_{bj}-\theta_j)/\theta_j\). Relative bias included between -0.1 and 0.1 are considered acceptable.
The Mean Square Error (MSE), defined as \(\sigma_j^2 + [1/B \sum_b (\hat{\theta}_{bj}-\theta_j)]^2\), with \(\sigma_j^2\) being the variance of the estimates across replications. The lower the MSE, the closer is forecast to actual.
The Skew-Normal
![]()
Scenarios
![]()
